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ABSTRACT 


In  this  paper  an  algorichm  it  proposed  for  solving 
continuous  linear  optimal  control  systems  with  state  space 
constraints  by  solving  a  sequence  of  linear  optimal  control 
systems  without  state  space  constraints.  The  convergence 
of  the  algorithm  is  proved  by  method  similar  to  cutting 
plane  algorithm  for  convex  programs  in  Banach  Spaces.  It 
is  also  shown  how  to  solve  the  problem  by  using  mathematical 
programming  algorithm  on  the  discretized  problem.  A 
numerical  example  is  solved  by  discretization  and  mathe¬ 
matical  programming. 


CUTTING  PLANE  ALGORITHMS  AND  STATE  SPACE 
CONSTRAINED  LINEAR  OPTIMAL  CONTROL  PROBLEMS 

by 

K.  C.  Kapur  and  R.  M.  Van  llyke 

1.  INTRODUCTION  AND  SUMMARY 

The  paper  breaks  naturally  into  three  parts.  The  first  part  consists  of 
Section  2  in  which  the  Cheney-Goldstein ,  Kelley  cutting  plane  algorithm  for  convex 
programming  problems  is  reviewed  and  Section  3  where  the  algorithm  is  generalized 
to  Banach  Spaces.  Theorem  3.1  is  new,  Theorem  3.2  is  a  special  case  of  Theorem  10.2 
of  Levitin  and  Polyak  [15],  Theorem  3.3  is  a  slight  generalization  of  a  theorem 
due  to  Topkis  [20] . 

The  second  part  consists  of  Section  4  and  5  where  a  scheme  for  solving 
continuous  linear  optimal  control  problems  with  state  space  constraints  by  solving 
a  sequence  of  problems  without  state  space  constraints  is  proposed.  The  convergence 
of  the  scheme  is  proved  by  identifying  it  as  the  cutting  plane  method  of  Section  3 
applied  to  the  optimal  control  problem.  This  scheme  assumes  one  knows  how  to 
solve  linear  control  problems  without  state  space  constraints  and  is  relatively 
independent  of  what  method  for  solving  the  problem  without  state  space  constraints 
is  used.  What  happens  to  our  proposal  when  the  unconstrained  problem  is  solved 
by  using  mathematical  programming  algorithms  on  the  discretized  problem  is  analyzed 
in  the  third  part,  Section  6.  It  is  shown  there,  using  a  slight  generalization  of 
the  results  of  J.  Cullum  [6],  that  as  the  mesh  size  of  the  discretization  approaches 
zero  the  answers  to  the  discrete  problems  converge  on  a  subsequence  to  a  solution 
of  the  original  continuous  problem.  Finally,  in  Section  7  a  numerical  example  is 
worked  out. 
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2 


2.  THE  CUTTING  PLANE  ALGORITHM  IN  FINITE  DIMENSIONAL  SPACES 

Kelley  [11]  and  Cheney-Goldstein  [5]  independently  proposed  a  cutting  plane 
algorithm  for  convex  programming.  The  problem  they  consider  is  to 


(1) 


Minimize  cx 
x 

Subject  to  g(x)  <  0 


where  c  -  (c^,  c^)  and  x  ■  (x^,  . ..,  xfl)  are  n  vectors  of  real  numbers 

and  g(x)  is  a  real  valued  convex  function.  If  g  is  differentiable  ac  x° 
with  gradient  g'(x°)  -  (x°) ,  ....  |*-  (x°)j  then 

g(x)  >  g(x°)  +  g'(x°)[x  -  x°]  .  Suppose  now  x°  i  K  -  {x  |  g(x)  <  0}  ;  i.e., 
g(x°)  >  0  then  if  g(x)  <  0  we  must  have 


(2)  g(x°)  +  g' (x°)[x  -  x° ]  -  g'(x°)x  +  [g (x°)  -  g'(x°)x°]  <  0  . 

Notice  that  the  above  inequality,  linear  in  x  ,  is  not  satisfied  for  x  *  x°  , 
since  g(x°)  >  0  .  If  K  is  bounded  this  suggests  the  following  algorithm.  Since 

K  is  bounded  we  can  find  a  matrix  A  and  vector  b  so  that  S  ■=  {x  I  Ax  <  b) 

■ 

is  compact  and  contains  K  .  Then  the  algorithm  is: 

Step  0: 

Solve  the  linear  program 


(3) 


Minimize  cx 
Subject  to  Ax  <  b  . 


Let  x  be  an  optimal  solution  to  (3)  and  set  k 


e* 


Step  1: 


k  k 

If  g(x  )  <  0  ,  x  is  an  optimal  solution  to  (1).  If  not  add  the  inequality 
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At  each  iteration  at  Step  2  we  minimize  cx  over  a  set  contained  in  S  and 
containing  K  .  At  every  iteration  this  set  gets  smaller  so  that  in  some  sense 
xk  approaches  K  .  More  precisely  we  have: 

Theorem  1  (Convergence) : 

Suppose  in  (1)  that  g  is  continuous  and  has  uniformly  bounded  gradient  on 
S  ;  i.e.,  there  exists  M  >  0  such  that  ||g'(x)||  <  M  for  all  x  e  S  .  Then 
the  sequence  {x  }  of  points  generated  by  the  cutting  plane  algorithm  has  a 

/  kil 

convergent  subsequence  and  for  any  convergent  subsequence  (x  f  converging  to 

A  A 

say  x  we  have  that  x  solves  (1) . 


Proof : 


Since  S  is  compact  any  sequence  of  points  in  S  will  have  a  convergent 


IM 


subsequence;  in  particular,  the  sequence  {x  }  is  contained  in  S  .  Let 

t  A 

be  any  convergent  subsequence  of  the  {xj  ,  and  let  z  »  Inf  {cx  |  g(x)  <  0  ,  x  e  S) 

Since  by  (2)  any  x  which  satisfies  g(x)  <  0  satisfies  the  constraints  of  (5), 

k  k  k 

i  i  i+1 

cx  <  z  for  any  i  .  Moreover  cx  <  cx  since  the  constraint  set  of  (5) 

gets  no  larger  as  k  gets  larger.  Hence  by  the  continuity  of  cx  , 

k  ^  AAA  A  AA  AA 

cx  ■+■  cx  <  z  .  If  x  e  K  ,  i.e.  ,  g(x)  <  0  then  cx  >  z  and  hence  cx  *  z 

•  A 

and  x  is  an  optimal  solution.  We  now  show  that  x  e  K  .  From  (5)  we  have 

/  m  i  Mr  ki  kii 

g\x  /  <  g'\x  /Lx  -  x  JJ  for  any  j  >  i  .  Letting  both  j  and  i  go  to 

infinity  with  j  >  i  we  obtain 


gOO  <  o 


r  k  m  /  k\  ( 

since  Lx  J  -  x  J  forms  a  Cauchy  sequence  converging  to  zero  and  ||g'lx  J\  |  <  M 
In  the  next  section  we  generalize  this  approach  to  function  spaces. 
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3.  A  CUT1 VNG  PLANE  ALGORITHM  IN  FUNCTION  SPACE 
The  problem  we  consider  here  is 

Minimize  c(u) 

^  Subject  to  g(u)  <  0 

u  e  S  c  U 

where  U  is  a  Banach  Space,  c(u)  a  lower  semi-continuous  convex  functional  on 

U  ,  and  g(u)  a  lower  semi-continuous  convex  functional  on  U  .  Finally  S  is 

convex  and  weakly  sequentially  compact. 

Before  we  go  through  the  details  of  the  cutting  plane  algorithm  for  this 

problem  it  if  perhaps  worthwhile  to  consider  the  statement  of  (1)  in  a  little 

more  detail.  Defined  on  a  Banach  Space  U  is  a  norm  |  |*|  |  such  that  ■+  x° 

if  and  only  if  | |xk  -  x° | |  -*•  0  .  Using  this  notion  of  convergence  g(u)  is 

lower  semi-continuous  (£.s.c.)  on  a  set  S  if  and  only  if  {u  |  g(u)  <  a}  is 

closed  for  all  a  .  g(u)  is  convex  on  a  convex  set  S  if  and  only  if 

{(y,u)  |  y  >  g(u)  ,  u  e  S}  is  convex  in  (R  *  U  . 

Besides  the  convergence  defined  by  the  norm,  usually  called  strong  convergence, 

there  is  the  notion  of  weak  convergence.  Associated  with  Banach  Space  U  there 

* 

is  the  adjoint  space  U  which  consists  of  all  continuous  linear  functionals 

*  *  .  *  . 

from  U  to  the  real  line.  Thus  if  u  e  U  we  will  denote  by  vu  ,u>  the  value 

ft  k  o 

of  u  at  u  .  We  say  that  a  sequence  {u  }  converges  weakly  to  u  ,  written 

u^  5  u°  if  for  each  continuous  linear  functional  u*  e  U*  (u* -*•  <u*,u°^  . 

Armed  with  the  notion  of  weak  convergence  we  can  define  a  set  T  to  be  weakly 

closed  if  {u^}  is  a  sequence  of  T  weakly  convergent  to  u°  implies  u°  c  T  . 

Since  strong  convergence  implies  weak  convergence  a  set  which  is  weakly  closed  is 

a  postevi.ovi  strongly  closed  but  not  necessarily  the  converse.  We  can  then  define 

g(u)  to  he  weakly  lower  semi-continuous  (w.&.s.c.)  on  S  if  (u  e  S  |  g(u)  <  a) 

is  weakly  closed  for  all  a  .  Clearly  any  continuous  linear  functional  is  weakly 


lift  tTw^Lj 
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continuous.  It  is  an  Interesting  property  of  convex  functions  that  if  they  are 
strongly  i.s.c.  they  are  w.JL.s.c.  The  proof  depends  on  the  following  result  from 
functional  analysis. 

Lemma  1: 

A  convex  subset  of  a  Banach  space  which  is  strongly  closed  is  weakly  closed. 

Proof ; 

Dunford  and  Schwartz,  p.  422 ,  [10]. 

Lemma  2; 

Let  g(u)  be  a  i.s.c.  convex  function  on  a  closed  convex  subset  of  a  Banach 
space  U  .  Then  g(u)  is  w.it.s.c. 

Proof : 

{u  |  g(u)  <  c}  is  closed  for  all  c  .  {(y,u)  |  y  >  g(u)  ,  u  e  S}  is  convex 
Hence  {u  |  g(u)  <  c)  is  also  convex.  Therefore  {u  |  g(u)  <  c}  is  weakly 
closed  for  all  c  by  Lemma  1.  Hence  g(u)  is  w.i.s.c. 

Finally,  we  say  that  S  is  weakly  (sequentially)  compact  if  any  sequence  in 
S  has  a  weakly  convergent  subsequence  to  a  point  in  S  . 

One  last  point  we  need  to  generalize  is  the  notion  of  a  gradient.  In  our 
applications  g  will  not  necessarily  be  differentiable.  A  weaker  concept  called 
subdlf ferentiability  suffices.  Figure  2  illustrates  the  concept  for  a  real 
valued  convex  function  of  one  variable.  The  subgradient  of 


■BrSfrjftKdSH*.  «**»*»* 1; 


(x) 


FIGURE  2 


The  slope  of  each  of  the  indicated  lines  is  a 
subderivative  of  f  at  x° 


g  at  u°  ,  9g(u°)  is  given  by  3g(u°)  -  {g  e  U  |  g(u)  >  g(u°)  +  <g  ,u  -  u°) 
for  all  u  e  (i)  .  If  g  has  a  gradient  (Frechet  derivative)  g'(u°)  at  u°  then 
3g(u°)  ■  (g'(u0)}  (Br^nsted-Rockafellar  [4]).  We  are  now  ready  to  discuss  the 
generalized  cutting  plane  algorithm. 


Step  0: 


We  can  solve 


(2) 


Minimize  c(u) 
Subject  to  u  e  S  , 


since  c(u)  is  weakly  lower  semi-continuous  by  Lemma  2  and  S  is  sequentially 
weakly  compact.  Call  the  solution  u°  ;  let  k  -  0  .  Go  to  Step  1. 


Step  1: 


Jr  V 

If  g(u  )  <  0  ,  u  is  the  desired  solution  to  (1).  If  not  add  the  inequality 
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(3) 


/  *k  v  y  *k  kv  .  k. 
<g  |U>  <  <g  .u  >  -  g(u  ) 


*k  k 

to  (4)  and  go  to  Step  2,  where  g  c  3g(u  )  .  Go  to  Step  2. 


Step  2: 

Solve 


(4) 


Minimize  c(u) 

Subject  to  u  e  S 

(g  °»u>  <  <g  °,u°>  -  g(u°) 


/  *k  v  /  *k  kv  ,  k. 
<g  ,u>  <  \g  ,u  /  -  g(u  ) 


k+i 


obtaining  as  an  optimal  solution  u  ,  set  k  -  k  +  1  and  go  to  Step  1. 

Theorem  1: 

Suppose  in  addition  to  the  previous  hypotheses  for  (1) ,  we  assume  that 

/  *k\°°  *k  k  *k 

l®  ;k«l  comPact  t*ie  strong  topology  and  g  e  3g(r  )  for  each  g 

used  in  the  cutting  plane  algorithm.  Then  the  sequence  {u  }  of  points  generated 

by  the  algorithm  contains  a  weakly  convergent  subsequence  and  for  any  weakly 


convergent  subsequence 


Proof: 


converging  to  u  ,  u  solves  (1) . 


Since  S  is  weakly  sequentially  compact  the  sequence  {u  }  will  have  a 

/  M 

weakly  convergent  subsequence.  Let  (u  f  be  a  weakly  convergent  subsequence 

A  A 

that  converges  to  u  .  Let  z  »  inf  {c(u)  |  g(u)  <0  ,  u  c  S}  .  For  each  i  , 

/  M  -  /  - 

c\u  J  <  z  and  hence  lim  sup  c\u  /  <_  z  .  Also,  by  the  lower  semi-continuity  of 

(  M 

c(u)  ,  we  have  lim  inf  c\u  /  >_  c(u)  .  Hence 
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(  ki )  (  ki)  * 

c(u)  <_  lim  inf  c\u  /  ^  lim  sup  c\u  /  <_  z  .  Now  we  will  show  that 

g(u)  <  0 

A  l»  A  A 

and  hence  c(u)  >_  z  so,  c(u)  -  z  . 

By  (4) 


*k.  k.  *k.  k.  I  k.\ 
<g  i,u  J>  <  <g  i,u  i>  -  g^u  1) 


or 


,(>)  «  ^ 


>  for  j  >  i  . 


**i  * 

Now  we  extract  a  subsequence  SL ^  from  the  sequence  k^  such  that  g  -*•  g 

*  1^1  *"1+1 
strongly  for  some  g  ;  this  we  can  do  by  compactness.  Let  v  •  u  -  u  , 


i  w 

v  -*■  0  and  g 


I  M  *£i  i  *£i  *  i  *  i 

\u  /  <  <g  ,v  )  -  <g  -  g  ,V  >  +  <g  ,V  y  .  Letting  i  -*■  “  we  have 


lim  (g  ,v*^  «  0  since  v1  ^  0 

i-HX. 


i  w 


lim  |<g  1  -  g*,vi>|  <  lim  ||g  1  -  g*  |  |  |  ]  v1 1  j  -  0 

1-Kt> 


*£i  *  i 

since  g  -*•  g  strongly  and  |  |v  |  |  is  bounded  [Dunford  and  Schwartz,  p.  68] 
On  the  other  hand  by  weak  lower  semi-continuity 


/  *£  £.  £.  , 
g(u)  <  lim  inf  g\u  /  <  lim  inf  Kg  ,u  -  u  1  ^ 


-  0  . 


Sometimes  we  can  guarantee  the  strong  convergence  of  the  u  rather  than 
weak  convergence  on  a  subsequence. 


10 


Definition: 

A  convex  function  c(u)  defined  on  a  convex  subset  K  of  a  Banach  Space 

U  is  uniformly  convex  if  there  exists  a  monotone  function  5 (t )  on  t  e 

12  12 

with  6 (t )  >  0  for  t  >  0  such  that  for  all  u  e  K  ,  u  e  K  with  u  ^  u 
there  exists  X  e  (0,1)  such  that 

(1  -  X)c(u^)  +  Xc(u^)  >  c((l  -  X)u*  +  Xu^)  +  6(||u*  -  u^]j)  . 

Telser  and  Graves  [18],  Levitin  and  Polyak  [15]. 

Theorem  2: 

If  in  addition  to  the  hypotheses  of  Theorem  1,  c(u)  is  uniformly  convex 
then  u^  converges  strongly  to  u°  which  corresponds  to  the  unique  optimal 
solution  of  (1). 

Proof: 

1  2 

First  we  establish  the  uniqueness  of  the  solution.  Suppose  u  ?r.d  u 
X  12 

optimizes  (1).  Then  u  *  (1  -  X)u  +  Xu  satisfies  the  constraints  for  all 

1  2 

X  e  (0,1)  and  if  u  /  u  by  uniform  convexity  for  some  X  e  (0,1) 

X  12  12 

c(u  )  <  (1  -  X)c(u  )  +  Xc(u  )  *  c(u  )  =  c(u  )  but  this  contradicts  the  optimality 
1  2 

of  u  and  u  so  they  must  be  the  same  point.  Thus  the  optimal  solution  i? 
unique.  Now  let  u  be  the  iterates  defined  by  our  algorithm.  As  k  -*■  ® 

Jq  —  mm  — 

c(u  )  +  c  for  some  c  .  If  the  original  problem  is  feasible  is  finite.  For 

T  >  0  let  e  ■  6(t)  >  0  .  Choose  K  sufficiently  large  so  that  for  n  k  >_  K  , 
n  k 

c(u‘)  -  c(u  )  <  e  .  For  any  n  k  >_  K  we  can  choose  by  uniform  convexity 
X  e  (0,1)  such  that 

c (u*)  *>  c((l  -  X)u^+  un)  <  (1  -  X)c (u^)  +  Xc(un)  -  6  ( 1 1 -  un{[) 

<  c(un)  -  6 (] |uk  -  un| | ) 
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the  last  Inequality  resulting  from  the  fact  that  n  ^k  implies  c(un)  >_  c(u^)  . 

k  X  X 

On  the  other  hand  c(u  )  <  c(u  )  since  u  is  feasible  for  the  problem  solved 
at  the  kth  iteration.  Thus  6(||u^  -  un||)  <  c(un)  -  c(u^)  <  c  which  implies 
| |un  -  uk||  <  x  .  Thus  ||un  -  u^||  is  a  Cauchy  Sequence  and  converges  strongly 
to  a  limit.  | | 

Another  consequence  of  uniform  continuity  is  that  we  do  not  need  to  keep  all 
the  added  constraints  in  the  algorithm.  This  is  important  for  computational 
efficiency.  We  revise  our  algorithm  in  the  following  way: 

Revised  Algorithm: 

Step  0: 

Solve 


(5) 


Minimize  c(u) 
Subject  to  u  e  S 


for 


1 

u 


let  k 


1 


Go  to  Step  1. 


Step  1: 


Ic  Ic 

If  g(u  )  <  0  ,  u  is  optimal.  Stop.  If  not  add  the  inequality  constraint 


(6) 


<g  k.u>  <  <g  \u*>  -  g(u*) 


*k  k> 


k 

to  the  system  in  Step  2,  where  g  e  3g(u  )  .  Go  to  Step  2. 


Step  2: 

Add  k  to  ^  to  obtain  ,  where  is  the  set  of  indices  of  the 

constraints  added  in  Step  1  retained  at  the  kth  iteration. 


■  urn******- ' 
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(7) 


Minimize  c(u) 
Subject  to  u  e  S 
*1 


t,u>  <  <g  V>  -  g(u*)«,  e  L 


k+1 

to  obtain  u  .  Delete  all  l  from  L,  for  which 

k 


s  *1  k+1  v  /  *1  ,  is 

kg  ,u  /  <  kg  ,u  ?  -  g(u  ) 

Set  k  «=  k  +  1  .  Go  to  Step  1. 


This  is  the  same  as  the  previous  algorithm  except  in  Step  2  all  the  inequalities 

k+1 


holding  strictly  for  the  optimal  solution  u 
following  result  due  to  Topkis  [20]: 


are  eliminated.  We  then  have  the 


Theorem  3: 

If  the  hypotheses  of  Theorem  2  are  satisfied,  the  revised  algorithm  generates 
u^  which  converge  strongly  to  u°  which  is  the  unique  optimal  solution  for  (1) . 


Proof : 

k  k+1  k 

c(u  )  <  c(u  )  for  all  k  thus  c(u  )  -*•  c  for  some  c  which  is  finite  if 

* 

the  original  problem  is  feasible.  Let  c  be  the  value  of  c(u)  for  an  optimal 

-  * 

solution  of  (1) .  Then  c  <  c  .  Since  S  is  weakly  sequentially  compact  and  all 

k  ”  ki  ki  o 

the  x  e  S  ,  there  is  a  subsequence  x  on  which  x  +  x  for  some  x°  e  S  . 

O  -  * 

By  the.  weak  lower  serai-continuity  of  c(u)  we  have  c(u  )  <  c  <  c  .  If  we  can 

show  g(u°)  <  0  we  have  c(u°)  >  c  and  thus  c(u°)  =  c  and  u°  is  optimal. 

Let  us  now  prove  this.  From  the  description  of  the  revised  algorithm  it  always 
k+1 

follows  that  u  satisfies  the  constraints  of  (7)  at  iteration  k  ;  in 
particular  we  have: 


(8) 


\ 

'  k.+l 

k. 

.  * 

l 

i 

h\ 

u 

-  u 

<  0 
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By  uniform  convexit”  for  each  i  there  exists  X^  e  (0,1)  such  that 


k.+l  k  *1  /  k  +1\  /  k A  /  k.+l  k  \ 

(9)  c  (1  -  Xi)u  1  +  Xiu  1  <  (1  -  X^) c yu  1  J  +  X1c\u  /  -  6^|  |u  -  u  ||] 


k  i+1  k 

But  (1  -  X^)u  +  X^u  is  feasible  for  (2)  at  iteration  k^  thus 


(10) 


On  the  other  hand  c 


,(.'*)  ..[<■•  y."‘“  .  . 


SO 


u-v«(»ki+i)+x1«(>).s(ii:i+i-:Mi) 

s  'M  - 


ki+i  .  uki 


Thus  from  (9)  and  (10) , 


SO 


(11) 


k.+i  k. 

|u  -  u  I  I  ■*  0  . 


Thus  from  (6) 


l(u  *)  +  | |3s(u  *)l I  I |u 


k.+i  k 

1  -  «  ‘l  I  <  0  . 


converges 


Letting  i  -*■  «  and  observing  that  at  least  on  a  subsequence  1 1  3g\ 

I  ki\  o 

strongly  to  some  number  we  have  g(u°)  <  lim  glu  I  <  0  .  Thus  u  is  an  optimal 
solution  to  (1).  The  proof  of  uniqueness  is  the  same  as  for  Theorem  2.  To  prove 


* 


isti*nnu>'i  m  w 
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ki+1  o 

strong  convergence  replace  u  by  u  starting  with  (9).  The  same  step 

yields 


(12) 


which  corresponds  to  (11). 


m’i'  > 
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4.  OPTIMAL  CONTROL  PROBLEMS  WITH  STATE  SPACE  CONSTRAINTS 

In  this  section  we  consider  (1) ,  In  the  next  section  various  modifications 
and  generalizations  of  this  basic  problem  will  be  considered: 


(la) 

Minimize  C(u) 

-  c  (u  ,x  (u) ) 

(lb) 

Subject  to  x(t) 

-  A(t)x(t) 

+  B (t)u(t) 

a.e.  [0,T] 

(lc) 

x(0) 

-  5 

(Id) 

x(T) 

T 

e  X 

(le) 

u(t) 

e  U(t) 

a.e.  [0,T] 

(If) 

D(t)x 

(t)  <  d(t) 

a.e.  [0,T] 

where 

x ( t )  *  (x1(t),  ....  xn(t))  is 

the  state 

vector,  C  is  a 

given  initial 

condition  for  x  ,  u(t)  ■  (Uj(t),  . ..,  u^(t))  is  the  control  vector,  A(t)nXn  , 

B(t)n*r  ,  D(t)mXn  ,  d(t)mxl  are  matrix  valued  functions  of  t  e  [0,T]  with  the 

'  T  n 

dimensions  indicated,  and  U(t)  is  a  set  valued  function  of  time.  X  C  E  . 

C(u)  -  c(u,x(u))  :  L ^ [ 0 , T ]  -*>  E*  is  convex  and  lower  semi-continuous  and  x(u) 
is  the  solution  of  (lb)  corresponding  to  u  The  initial  assumptions  we  make 
on  (1)  are: 

Assumptions : 

Assumption  Al: 

A(t)  ,  B(t)  ,  D(t)  ,  d(t)  are  measurable  on  [0,T]  and  uniformly  bounded. 
Assumption  A2: 

u(t)  is  measurable  on  [0,T]  . 

Assumption  A3: 

U(t)  is  compact  and  convex  for  each  t  and  continuous  in  t  with  respect  to 
the  Hausdorff  metric. 


LA*  -  — *  '  ‘ 
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Assumption  A4: 

T 

X  Is  closed  and  convex. 

First  we  will  give  the  algorithm  for  (1)  and  then  do  the  analysis  which  shows 
it  is  the  cutting  algorithm  of  Section  3  applied  to  (1).  Since  we  are  mainly 

j 

Interested  in  handling  state  space  constraints  we  will  assume  we  know  how  to 
solve  the  problems  of  the  form  (1)  without  state  space  constraints;  namely,  that 
we  can  solve  the  problem  defined  by  (la),  (lb),  (lc) ,  (Id)  and  (le).  Methods  for 

t 

solving  this  later  type  of  problem  can  be  found  in  [2,  8,  9,  13].  The  most  natural 
of  these  from  our  point  of  view  is  the  approach  due  to  Dantzig  [Dantzig,  [9]], 
[Dantzig  and  Van  Slyke,  [8]].  These  are  somewhat  difficult  to  apply  if  c(u,x) 
depends  on  x  explicitly. 

Algorithm: 

I 

Step  0: 

Find  optimal  solutions  u^(t)  ,  x*(t)  for  the  problem  defined  by  (la)  to 
(le).  Set  k  -  1  . 

j 

i 

Step  1: 

if 

If  D(t)x  (t)  <  d(t)  a.e.  [0,T]  then  the  current  solution  is  optimal.  If 

not  add  x  ..  to  the  state  vector  where 
n+k 


dx 


(2) 


dt 


—  (t)  -  [Tr(t;uk)D(t)]x(t)  -  [Tr(t;uk)d(t)] 


where  x(t)  ■  (x^(t) . xn(t))  and  where  we  impose  the  boundary  conditions 

k 

xn+k(0)  -  0  ,  xn+k(T)  <  0  .  Tr(t,u  )  -  (tt^ ,  ...,  ir^)  is  given  by 


(3) 


TTi(t,U  ) 


,1  if  Di(t)xN(t)  -  d±(t)  >  0 


0  if  Di(t)x‘"(t)  -  dt(t)  <  0 
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for  all  t  e  [0,T]  and  i  -  1,2,  m  .  Set  k  -  k  +  1  . 


Step  2; 


Find  optimal  solutions 


k+1,  . 
u  (t) 


for 


(4a) 

(4b) 

(4c) 

(4d) 

(4e) 

(4d) 


Minimize  C(u) 

Subject  to  x(t)  ■  A(t)x(t)  +  B(t)u(t)  a.e.  [0,T] 

XR+j(t)  -  [ir(t;u^)D(t)  ]x(t)  -  ir(t;u^)d(t)  a.e.  [0,T]  , 
j  ■  1,  . . . ,  k  . 

x(0)  -  5  ,  x  (0)  -  0  j  -  1 . k  . 

n+j 

x(T)  e  XT  ,  xn+j(T)  <0  j  -  1 . k  . 

u(t)  e  U(t)  a.e.  [0,T]  . 


Go  to  Step  1. 


Note  that  at  each  iteration  of  the  above  algorithm  we  solve  a  problem  without 
state  space  constraints.  The  cost  of  avoiding  such  constraints  is  that  the  state 
space  vector  increases  by  one  dimension  each  iteration.  The  convergence  properties 
of  the  algorithm  is  given  in  the  following  three  theorems.  They  are  proved  by 
identifying  the  algorithm  as  a  special  case  of  the  cutting  plane  algorithm  described 
in  Section  3. 

Theorem  1: 

k  k 

Under  Assumptions  A1  to  A4  the  algorithm  generates  a  sequence  u  ,  x  which 

ki  ^i  ki  ki 

contains  a  subsequence  u  ,  x  on  which  u  converges  weakly  and  x 

k  k 

converges  strongly.  Moreover,  for  any  subsequence  such  that  u  1  -*■  u°  ,  x  1  -*•  x° 

w 

we  have  that  x°  ,  u°  solves  (1) . 


Tneorem  2: 


If  in  addition  to  the  hypotheses  of  Theorem  1,  Assumption  A5  is  satisfied  then 
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the  iterates  u  ,  x  given  by  the  algorithm  converge  strongly  to  a  unique 
solution  u°  ,  x°  for  (1). 


Assumption  A5: 

C(u)  is  uniformly  convex  in  u  for  all  u  satisfying  (le) . 

Theorem  3: 

k 

Under  the  same  hypotheses  as  Theorem  2  the  iterates  {u  }  obtained  from  the 
following  algorithm  obtained  by  eliminating  the  strict  inequalities  converges 
strongly  to  the  unique  optimal  solution  of  (1). 


Revised  Algorithm: 

Step  0. 

Solve  the  problem  defined  by  (la)  to  (le)  for  u*(t)  ,  x^(t)  .  Set  k  =  1  . 
Go  to  Step  1. 

Step  1: 

If  D(t)x(t)  <  d(t)  a.e.  [0,T]  then  the  current  solution  is  optimal.  If 

not  add  x  to  the  state  vector  (add  n  +  k  to  J.  where  J.  is  the  set  of 
n+k  k  k 

Indices  of  the  state  space  co-ordinaLes  retained  at  the  kth  iteration)  where 


dx 


n+k 


dt 


(t)  «  [ tt ( t ; uk ) D ( t )  ]x(t)  -  [7r(t;uk)d(t)] 


with  boundary  conditions  x  ^(0)  ■  0 
Go  to  Step  2. 

Step  2: 

k+1 

Find  optimal  solutions  u  (t) 


,  Xn+k(T)  <  0  .  TT(t;uk)  is  given  by  (3). 


k+1 

x  (t)  for  the  problem  defined  by  (4a)  , 


(4b) ,  (4f)  and 
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(5a) 

• 

X 

n+j 

■  [n(t;ubD(C))x(t) 

-  n(t;u^)d(t) 

a.e.  [0 ,T ]  j  e  JR 

(5b) 

x(0) 

■ 5  •  wo)  ■ 0 

j  e  Jk 

(5c) 

x(T) 

6  XT  .  *n+J (I)  :  0 

j  e  Jk  • 

Delete  all  j 

from  J. 

k 

k+1 

for  which  x  ,j(T)  < 
n+j 

0  to  8et  Jk+1 

.  Set  k  *  k  +  1  . 

Return  to  Step  1. 


A  commonly  occuring  example  of  a  C(u)  which  is  uniformly  convex  in  u  is 
the  following 


C(u) 


c|$(t;0K  + 


J 


$(t;s)B(s)u(s)ds 


where 


T 

c(x,u)  =  J  [x(t)Q(t)x(t)  +  u(t)P(t)u(t)  +  q(t)x(t)  +  p (t)u(t) ]dt 
0 

and  Q(t)  is  symmetric  and  positive  semi-definite  for  each  t  e  [0,T]  ,  P(t)  is 

uniformly  positive  definite  that  is,  inf  u(t)p(t)u(t)  >  o  >  0  for  all 

l|u(t)||-l 

t  e  [0,T]  ,  and  every  component  of  Q(t)  ,  P(t)  ,  q(t)  ,  p(t)  is  square  integrable 
on  [  0 ,  T  ]  . 

Let  us  now  identify  the  relations  between  the  optimal  control  problems  (1) 
and  the  generalized  cutting  plane  method  described  in  Section  3.  To  do  this  we 
first  integrate  (lb);  this  results  in 


/ 


<J>(t;s)B(s)u(s)ds 


(6) 


x  (t)  =  <Kt;0)£  + 


20 


where  <t(t;s)  is  the  matrix  solution  of 


—  $(t;s)  **  A ( t ) (t  ;s) 


with  the  boundary  condition 


4>  (s  ;  s )  =  I 


This  can  always  be  carried  out  under  A1  and  A2 .  Moreover  4>(t;s)  is  absolutely 
continuous  and  uniformly  bounded  in  s  and  t  satisfying  0  <  t  <  T  *  0  *  s  ^  ^ 
(Lee  and  Marcus  [13]). 


Lemma  1: 


u(t) | |  |  u ( t )  e  U(t)  ,  t  e  [0,T] }  <  °°  ;  i.e.,  the  controls  are 


uniformly  bounded.  Also 


sup  <  I  I  x  (t )  (  I  I  x(t)  =  <p(t;0)4  +  I  $  (t;s)B(s)u(s)ds,t  e  [0,T],u(t)  e  U(t)>  <  ® 


Proof: 


Suppose  the  lemma  is  false.  Then  there  exists  a  sequence  u  e  iKt^.)  with 
-*■  t  k  [0,T]  and  |  j u^ j  J  ■*  +°°  .  But  by  the  Hausdorff  Continuity  of  U(t) 
for  any  e  >  0  there  is  a  5  >  0  such  th  .t  1 1^  -  t  |  <  5  implies  if  u  e  iKt^) 
then  inf  { |  ju^  -  u| |  |  u  e  U(tQ)}  <  e  .  But  sup  { |  |u| |  |  u  e  iKt^) }  is  finite 
so  that  lim  sup  { | |u  | | }  <  sup  { | |u| |  |  u  e  U(tQ)  )  +  e<“* 

I  Mt)  |  |  <  |  |*(t;0)  |  |  |U||  +  J  |  |$(t;s)  |  |  l  | B(s)  |  |  |  | u(s)  |  |  ds  . 

But  all  the  norms  on  the  right  of  the  above  expression  are  uniformly  bounded  so 
that  x(t)  is  uniformly  bounded.  |  | 


1 
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Let  S  -  {u(t)  |  u(t)  e  U(t)  for  t  e  [0,T]  and  u(t)  measurable}  . 

Lemma  2 : 

Under  A3,  3  is  convex  and  weakly  sequentially  compact. 

Proof : 

S  is  bounded  in  L^tO.T]  by  Lemma  1.  L^tOjT]  is  reflexive  so  that  S  is 
weakly  sequentially  conditionally  compact  by  Theorem  1.3.28  of  Dunford  and 
Schwartz  [10].  S  is  obviously  convex  and  strongly  closed  hence  by  Theorem  V.3.13 
of  Dunford  and  Schwartz  [10]  S  is  weakly  closed. 

Let  z(t)  =  4>(t;0K  and  define  the  operator  Y[u]  :  L^fO.T]  L^tO.T]  by 

t 

(9)  Y[u]  *  y(t)  -  j  $(t;s)B(s)u(s)ds  . 

0 

Thus  x(t)  =  z(t)  +  y(t)  where  y(t)  ■  Y[u](t)  .  The  linear  operator  is 
completely  continuous  (or  by  some  authors  compact)  if  it  maps  bounded  sets  of  the 
domain  into  conditionally  strongly  compact  sets  in  the  range  (Liusternik  and 
Sobolev,  p.  129,(17]).  The  main  consequence  is  for  reflexive  spaces  weakly  compact 
sets  are  mapped  into  compact  sets  (Dunford  and  Schwartz,  p.  539,  [10]). 

Lemma  3: 

Y  is  a  completely  continuous  operator  from  L^tO.T]  to  L^O.T]  . 

For  the  proof  of  Lemma  3  one  can  use  the  example  on  p.  131  of  Liusternik 
and  Sobolev  [17]  by  letting 

(t;s)B(s)  0  »  3  »  t  <  T 

0  otherwise  . 
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Corollary; 

Y[S]  ■  {Y[u]  |  u  e  S}  is  strongly  compact  and  convex. 

Let 


T 

g(u)  =  J  n(t;u)[D(t)x(t)  -  d(t) ]dt 
0 

(10) 

T 

m 

£  Max[0,D. (t)x(t)  -  d.(t)]dt 
=1  1  1 

where  ir(t,u)  is  defined  by  (3),  and  D^(t)  and  d^(t)  are  the  ith  rows  of 
D(t)  and  d(t)  respectively.  Clearly  g(u)  <0  if  and  only  if  (If)  is  satisfied. 

Lemma  4: 

g(u)  is  convex  on  S  .  It  is  strongly  continuous  on  S  and  therefore 
weakly  lower  semi-continuous. 

Proof : 

12X12  12 

Let  u  eS,u  eS,u  ■  (1-  X)u  +  Xu  for  0  <  X  <  1  .  Let  x  ,  x  , 

X  1  2  X 

and  x  be  the  state  trajectories  resulting  from  u  ,  u  ,  and  u  respectively. 

Then 
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r 


(1  -  A)g(u1)  +  Ag(u2) 


T 

J  I  j(l  “  A)Max  0,Di(t)x1(t)  -  d1(t)  +  AMax  0,Di(t)x2(t)  -  d± ( t )J ^dt 

0  1=1 

T 

J  l  Max|o,(l  -  A)^Di(t)x1(t)  -  d^tjj  +  A^D^tJx^t)  -  di(t)j|dt 
0  i=1 

T 

j*  £  Max|o,Di(t)xX  (t)  -  di(t)|dt 


g(uA)  . 


Thus  g  is  convex  on  S  .  The  operator  which  maps  u  -*■  x  =  Y(u)  +  z  is  continuous 
from  L^tO.T]  to  L^tO.T]  because  Y(u)  is  completely  continuous  and  therefore 
continuous.  The  mapping  x  -*■  6^(t)  =  Max[0 ,D^ (t)x (t)  -  d^(t)]  is  a  continuous 
mapping  from  L^tO.T]  to  L^fO.T]  under  Al.  Thus 


u(t)  -*■  l  Max[0,D  (t)x(t)  -  d  (t)  ] 
i-1 


is  continuous.  Finally  z  -*■  ^  l*z(t)dt  is  in  the  conjugate  soace  of  L™[0,T] 

0 

and  is  ther** f  n  r  p  continuous.  Thus  g(u)  is  continuous.  By  Lemma  3.2  g  (u)  is 
weakly  lower  semi-continuous. 


To  finish  up  the  identification  of  the  components  of  Theorem  3.1  we  must 
define  an  element  of  the  sub-differential  of  g(u)  at  a  point  u°  .  Let 


i 

<g  (u°) ,u>  =  J*  r  (t ;u°)D ( t)y ( t) dt 


where  y(t)  =  Y(u)  and  ”(t;u  )  is  given  by  (3) 


-TB.'JTTHTI — If  ■W’ 


Lemma  5: 


g  (u°)  given  by  (11)  belongs  to  3g(u°)  for  all  u°  e  S  . 


Proof: 


We  must  show 


g (u)  -  g(u°)  >  <g  (u°),u  -  u°> 


g(u)  -  g(u  )  *  J  {n(t;u)[D(t)[z(t)  +  y(t)]  -  d(t)] 

0 

-  Tr(t;u°)  [D(t)  [z(t)  +  y°(t)]  -  d(t)]}dt 


i 

J 


)((D<t)U(t>  +  y Ct) ]  -  d(t) ]  -  D(t)[z(t)  +  y°(t) ]  -  d(t))dt 


1 

J*  tt  (t;u°) 


)D(t) ] y ( t )  -  y  (t) ]dt 


<g  (u°),u  -  u°> 


where  z 

i 

since  tt 

i 

i 

i 

if  (t;u 

:  i 

1  ^ 

| 

i 

Finally: 

w 

f 

k 

Lemma  6 : 

o  oN 


*  o 
(g  (u°) 


o  r*  r 

u  e  S }  is  compact  in  the  strong  topology  of  [ 0 , T ]  =»  L, ^ 1 0 , T ] 


I 

i 


Proof: 


<g  (u°),u>  can  be  written  as  <(g  (u°)  ,u]>  =  ^G  (u°),Y(u))>  where 
G  (u°)  :  Y[s]  -*•  by  the  relation 

T 

(13)  <G  (u°) ,y>  =  j*  n  (t  ;u°)D(t)y(t)dt  . 

0 

Thus  for  each  u°  ,  G  (u°)  is  a  continuous  linear  functional  on  Y[S]  .  Y [ S )  is 

compact  by  the  Corollary  to  Lemma  3.  We  will  now  use  the  Ascoli-Arzela  Theorem 

o  |  o 

Dunford  and  Schwartz,  17.6.7,  [10]  to  show  that  the  family  G  =  {G  (u  )  |  u  e  S} 
is  compact  in  C[Y[S]]  ,  the  space  of  all  continuous  functions  on  Y[S]  to  . 
First  we  show  that  G  is  uniformly  bounded 


t  % 

|<G  (u°),y>|  =  |  ir(t;u°)D(t)y(t)dt[ 


0 

T 

<  J"  /m  5  y  dt  =  /in  D  yT 
0 

where  D  ^  |  | D ( t )  (  [  for  all  t  (Al)  y  >  !  |y(t)  |  '  for  all  t  and  all  u  £  S 
(Lemma  1)  and  |  |  tt  ( t ; u°)  |  ]  <  /in  (3).  Now  to  show  that  the  family  is  equi- 
continuous;  that  is,  for  all  e  >  0  we  must  show  there  exists  6  >  0  such  that 
if  !  ! |  I  <  6  then  |^G  (u°)  ,y^  -  y2]>|  <  e  for  all  u°  e  S  .  But 


T 

|<G  (u°)  .y1  -  y2>i  *  |  J  m(t  ju°)D(t)  (y1(t)  -  y2(t))dt| 

0 

<  |  | m (t ;u°)D(t)  !  !  | ly1  -  y2' ! 

<  v'a  D  c 


ol 


n* 

i2  [°>Ti 


anc 


the 


where  the  norms  a'-e 


L^[0,T]  respectively.  Thus  by 
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f 
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Ascoli-Arzela  Theorem  the  family  G  is  compact  in  C[Y[S]]  and  hence  in  [0,T] 

★  o  i  o  i 

Now  we  must  show  that  the  family  (g  (u  )  |  u  e  S}  is  compact  in  L?  .  To  do 


this  we  show  for  every  sequence:  u  c  S  there  exists  a  convergent  subsequence 

ltk.o  *  o  v 

which  we  will  also  denote  by  u  ,  u  -*•  u  such  that  \g  (u  )  -  g  (u  ),u/ 
converges  to  0  as  k  -*■  <*>  uniformly  for  all  u  such  that  |  |u|  |  <  1  •  But 
this  is  equivalent  to 

<G*(uk)  -  G*(u°),y(t)>  -  0 

uniformly  for  all  y  e  Y(B)  where  B  is  the  closed  unit  ball  in  L^iO.T]  .  Y[B] 
is  compact  by  Lemma  3  and  the  fact  L^iO.T]  is  reflexive  and  hence  B  is  weakly 


sequentially  compact  Dunford  and  Schuartz,  V.4.7,  [10],  Thus  in  particular,  Y[B] 

*  k  *  o 

is  bounded.  We  choose  as  our  subsequence  one  for  which  G  (u  )  -*■  G  (u  )  strongly 
Then  sup  Kg  (uk)  -  G  (u°),y(t)>  |  ||y(t)j|  <  K}  ->  0  for  any  fixed  K  . 


Proofs  of  Theorems  4.1,  A. 2  and  4.3: 

Theorems  4.1,  4.2  and  4.3  follow  immediately  from  Theorems  3.1,  3.2  and  3.3 
respectively  under  the  following  identifications: 


Section  3 
c(u) 

g(u) 


<g  (u°),u> 


Section  4 
C(u)  =  c (u , x (u) ) 


J  *<t, 


u)[D(t)x(t)  -  d(t) ]dt 


(u(t)  |  u(t)  e  U(t)  for  t  e  [0,T]  and 
u  is  measurable} 

T 


j  7l(t,UC 


)D(t)y(t)dt  . 


For  the  last  identification,  at  the  kth  iteration,  we  are  adding  the  constraints 


i 
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x 


n+lt 


(T)  <  0 


/>  k 

at  the  Step  2  in  Section  A.  We  have  x  ,  (T)  ■  J  ff(t,u  )D<t)y(t)dt  +  const,  term 

n+K  Q 

and  hence  the  addition  of  these  constraints  is  equivalent  to  constraints  added  in 
Step  1  in  the  algorithm  in  Section  3. 
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5.  MODIFICATIONS  AND  GENERALIZATIONS 

Several  generalizations  and  variations  can  be  made  to  the  basic  algorithm. 

First  we  will  discuss  the  question  of  variable  initial  points;  that  is  suppose  we 

replace  the  initial  condition  (4.1c)  x(0)  ■  (  by  (lc')  x(t)  e  X°  .  If 
T  T 

X  «  {x  }  is  a  single  point  then  the  same  proofs  work  for  convergence  just  by 

o  T 

running  time  "backwards."  In  the  more  general  case  where  neither  X  or  X  are 

o  T 

singletons  convergence  can  be  proved  where  both  X°  and  X  are  closed  convex 
o  T 

sets  and  either  X°  or  X  is  bounded.  We  will  not  carry  out  the  details  her-; 
because  the  proof  is  rather  messy.  The  main  modification  is  in  the  resulting 
analog  of  Theorem  3.2  for  uniformly  convex  objectives  in  u  .  Strong  convergence 
of  the  trajectories  in  this  case  may  occur  only  on  a  subsequence. 

Other  possible  applications  of  the  cutting  plane  algorithm  to  the  statu  space 
controlled  problem  are  also  possible.  These  can  be  derived  by  changing  g(u) 
from  the  one  defined  by  (4.10).  Also  more  general  constraints  on  the  state  than 
linear  ones  can  be  treated.  As  an  example  of  other  approaches  the  one  in  Levitin 
and  Polyak,  Section  10,  [15]  deserves  mention.  There  the  state  space  constraints 
(4. If)  are  replaced  by 

q(x(t))  <0  t  e  [0,T] 

where  q  is  a  real  valued  differentiable  convex  function  of  its  argument.  Then 
Instead  of  the  g(u)  defined  by  (4.10)  they  consider 

g(u)  -  max  q(x(t))  . 

0<t<T 

■  ■ 

The  cuts  corresponding  to  (3.3)  become 

q(xk(tk))  +  ,x(tk)  "  xk(tk^  -  0 

k 

where  x  is  the  kth  iterate  and  t^  is  given  by 
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g(xk(e.  ))  -  max  q(xk(c))  . 

'  '  0<t<T 

m  m 

However,  we  Chink  our  choice  leads  Co  compuCaCionally  simpler  algorichms. 

Now  ic  is  probably  proper  Co  Curn  Co  Che  quesCion  of  compuCaCion.  Solving 
problem  (4.4)  by  compuCer  will  inevicable  require  discretizacion  aC  some  poinC. 
In  Che  nexC  secCion  we  discreCize  from  Che  beginning  following  Van  Slyke  and 
Wees  [21].  We  Chen  use  some  resulcs  of  Cullum  [6]  Co  show  as  Che  discreCizadon 
mesh  gees  smaller  and  smaller  Che  soluCion  of  Che  discreCized  problem  approaches 
thaC  of  Che  conCinuous  problem. 
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6.  COMPUTATION'  BY  DISCRETIZATION 


Let  us  consider  the  problem 


T 


(la) 

Minimize 

C(u)  -  j*  [c(u,t)  +  h(t)x(t)]dt 

0 

(lb) 

Subject  to 

x(t)  *  A(t)x(t)  +  B (t) u (t)  a.e. 

[0  ,T  ] 

(lc) 

X 

/-N 

o 

V-/ 

1 

o 

(Id) 

x(T)  -  CT 

(le) 

E(t)u(t)  <  e(t) 

(If) 

D(t)x (t)  <  d(t) 

where  c(u,t) 

is  continuous 

and  convex  in  u  and  measurable  in 

t  ,  h(t) 

measurable  and  uniformly  bounded  in  t  on  [0,T]  .  U(t)  *  {u  j  E(t)u(t)  '  e(t)’ 
is  bounded  for  each  t  and  as  a  set  function  is  Hausdorff  continu  >;s ,  each  element 
of  E(t)  and  e(t)  is  in  L2[0,T]  and  is  uniformly  bounded  on  [0,Tj  ,  we  also 
assume  E(t)u(t)  <  e(t)  has  a  solution  for  each  t  .  A(t)  ,  B(t)  are  continuous 
on  [0,T]  .  This  problem  is  a  special  case  of  (4.1).  E ( t ) ^  r  ,  e(t)P  ^  are 
matrix  valued  functions  of  t  e  [0,T]  . 

To  discretize  we  first  divide  [0,T]  into  K  parts  each  of  length  A  =  T/K  . 
We  then  make  the  Cauchy-Euler  approximation 

.  k+1  k 

(2)  ff  (k«  =  *— 

lc 

where  we  define  x  ■  x(kA)  ,  k  -  0,1,  K  .  Many  other  approximations  fo  the 

derivatives <Todd  [19])  with  a  higher  degree  of  accuracy  can  be  analyzed  using  the 
same  approach  as  the  one  presented  here;  however,  the  notation  in  these  cases 
becomes  extremely  cumbersome.  Then  if  we  let  A  =  [I  +  AA(kA)]  ,  B  -  AF.(k‘.)  , 

E^  *  E(kA)  ,  e^  -  e(kA)  ,  =  D(kA)  ,  d^  *  d(kA)  ,  u^  =  u(kA)  ,  c^(u)  =  Ac(u,k‘.)  , 

and  h  (u)  *  h(u,kA)  (1)  can  be  approximated  by 
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(3a) 

(3b) 

(3c) 

(3d) 

(3e) 

(3f) 


Minimize  C(u)  ■  J  c,  (uk)  +  hkxk 

k-0 

Subject  to  xk+1  -  Akxk  +  Bkuk  k  -  0,1 . K-l 


Ekuk  <  ek  k  -  0,1,  ....  K-l 

Dkxk  <  dk  k  -  1,  ....  K-l  . 


This  is  a  mathematical  programming  problem  with  linear  constraints  and  a 
convex  objective  function  for  which  many  algorithms  exist.  See  for  example, 

Abadie  [1],  Cheney-Goldstein  [5],  Kelley  [11],  Klinzi  and  Krelle  [12],  Zoutendijk  [22] 
However,  (3)  has  very  special  structures  which  allows  much  simplification  [Dantzig 
and  Van  Slyke,  1969,  Section  4],  To  see  that  this  is  necessary  let  us  consider  the 
number  of  constraints  and  variables  in  (3).  There  are  Kn  equations  in  (3b), 
n  equations  in  (3c),  n  equations  in  (3d),  K  x  p  inequalities  in  (3e),  and 
(K-l)  x  £  inequalities  associated  with  (3f)  where  i  is  the  number  of  rows 
of  D  .  This  adds  up  to  K[n  +m+£+p]-£+2n  equations  in  nK  +  rK 
variables.  The  parameters  n  ,  l  ,  m  and  p  are  usually  relatively  small;  however 
K  determines  the  accuracy  of  the  approximation  (2)  and  could  be  extremely  large 
if  a  high  degree  of  accuracy  is  required.  In  order  to  reduce  the  problem  we  start 
by  analyzing  a  much  simpler  problem  than  (3)  and  then  solve  a  sequence  of  these 
easier  problems  in  order  to  solve  (3).  Suppose  we  consider  (3)  without  the  state 
space  constraints  (3f)  then  the  problem  can  be  simplified  considerably.  Let 


w 


*k-lAk-2 


I 


k  >  J 


this  satisfies 
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$ 

k+l,j 


I 


then  we  can  combine  (3b),  (3c)  and  (3d)  by  solving  (3b)  by  successive  substitution. 
This  results  in: 


(6) 


K,o 


r  + 


K 

l 

j=l 


$  BJ 

K.J 


•1  j-1 
u 


Then  the  system  (3)  ignoring  the  state  space  constraints  takes  on  the  much  simpler 


form 

k-1 

k  (  k 

(7a) 

Minimize 

I  c  (uk) 
k=0 

+  h  C°  +  y  ,B 

j  k,o  ^  k,j 

(7b) 

Subject  to 

K  4 

I  *K  / 

j-1  *’3 

-1  j-1  K  _o 

u  *  5  -  A 

K,o 

(7c) 

k  k  k 

E  u  <  e 

k  *  0,1,  . . . ,  K-1  . 

(7)  has  r  +  rK  equations  in  rK  variables.  As  a  result  of  having  removed 
the  state  space  constraints  (3f)  the  constraints  of  (7)  are  in  a  perfect  form  for 
Dantzig-Wolfe  Decompos ition ,  (Dantzig  [7);  Dantzig  [9];  Dantzig  and  Van  Slyke  [8]) 
since  the  constraints  on  the  controls  are  linked  in  time  only  through  the  relativel 
small  number  of  equations  in  (7b).  The  application  of  decomposition  techniques 
reduces  (7)  to  a  problem  which  looks  more  or  less  like  K  +  1  problems,  one  with 
r  equations  and  K  with  r  equations  which  are  solved  sequentially  instead  of 
(3)  which  is  one  big  problem  with  K[n  +  m  +  p]  +  2n  equations  (not  counting  the 
state  constraints)  which  is  solved  once. 

Now  we  return  to  pick  up  the  state  space  constraints  which  we  have  ignored  up 


until  now.  The  idea  is  the  same  as  in  the  continuous  case. 


4 
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Algorithm: 


Step  0: 


Solve  (7)  for  u*  -  (u^’1,  uK  ^’*)  ,  set  x  ■  1  go  to  Step  1. 


Step  1: 


If  for  k  =  1 . K 


_  K  I  O  ,  r  .  I 

D  h,o^  +  ^  Vj*  u 


k-1  k-.l,x 


>  <  dr 


u  is  optimal  if  not  set 


1  «  1  “ 


k-1  k-l,x 
« •  ' 


>  d 


0  otherwise 


k  k. 

for  i  *  1,  ....  I  ,  k  -  0,1,  ...,  K-1  where  and  d^  are  the  ith  rows  of 

Ic  k 

D  and  d  respectively.  Then  we  add  the  constraint  corresponding  to  the  new 
state  variable  Xn+T  given  by  (8)  to  Equation  (10c)  in  Step  2  and  procede  to 
Step  2 


(8a) 

(8b) 

(8c) 


k+1  k  .  k,x/T,k  k  ,k. 

x  -  x  +  ir  (D  x  -  d  ) 
n+x  n+T 


x  -  0 
n+x 


xK  <  0  . 
n+x  - 


k  -  0,1,  .  . .  ,  K-1 


Equation  (8)  is  equivalent  to  (9) 


(9) 


K-1  ,  ,  .  K-1  ,  , 

r  k,T_.k  k  r  k,T,k 

I  TT  DX  <  l  TT’d  . 


k-1 


k-1 


|C 

(9)  is  added  to  (10)  after  substituting  for  x  using  (6) 
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\ 


Step  2: 

Solve 


,  ( 

(  k 

(10a) 

Minimize 

l  +  h  | 

k  +  i  *k  ',3 

( k'°  j.i  ku 

(10b) 

Subject  to 

l  *. 

L  k,j 

•  -  5k  -  4,  5° 

k  ,0 

and 


(10c) 


Kr*  k»vr>kL  t°,  r  .  nk-l  k-l,v 

l  *  D  \\  0Z  +  L  K  u 

k=l  K’°  j-1  *’J 


K-l 


c  k,v,k  ,  , 

<  ]_  tt  d  for  o  =  l, 

=  k=l 


(10d) 


lr  V  If 

E  u  <  e  k  =  0,1,  ....  K-l 


The  convergence  of  this  algorithm  is  discussed  in  Van  Slyke  and  Wets  [21], 

k 

Moreover  if  c^/u  )  uniformly  convex  in  u  the  redurdant  inequalities  of  (10c) 

k 

can  be  dropped  by  Theorem  3.3.  If  c^(u  )  is  linear  for  each  k  the  algorithm 
terminates  in  a  finite  number  of  steps  and  again  redundant  constraints  may  be 
dropped  Van  Slyke  and  Wets  [21]. 

The  last  point  we  want  to  discuss  is  the  validity  of  the  discretization.  In 

particular  as  the  number  of  discretizations  K  goes  up,  do  the  optimal  solutions 

of  the  discrete  problems  approach  an  optimal  solution  of  the  original  problem  (i) i 

The  answer  is  yes.  The  proof  of  this  for  autonomous  problems  is  due  to  Cullum  [6] 

We  need  only  make  slight  modifications.  The  first  thing  we  need  are  assumptions 

which  guarantee  that  the  discrete  problems  (3)  have  solutions  for  K  sufficiently 

large.  We  make  the  following  assumption  which  is  somewhat  stronger  than  the  one 

made  by  Cullum.  Let  R  =  {z  |  3  x(t),u(t)  satisfying  (lb)  -  (If),  z  =  x(T)}  . 

T 

We  assume  that  £  is  an  interior  point  of  R  .  Then  the  results  of  Cullum 
apply  to  our  problem.  Cullum's  proof  of  this  fact  depends  on  the  uniformity 
condition  embedded  in  the  following  theorem: 


v.4W...  «. 
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Theorem  1: 


Let  u  (t)  be  any  uniformly  bounded  sequence  of  functions  on  [0,T]  which 


are  constant  on 


[T  \  T  T  2T  \  T K-l 

0,—)  ,  — 1 .  T,t  respectively 


for  K  »  1,2,  ...  Let  x  ,  k  *  0,1 . K  ,  K  *  1,2,  ...  be  the  solution  of 

k  K / kT \  A  K 

(3b,  3c)  corresponding  to  u  -  u  1—1  •  Let  x  (t)  be  the  piecewise  linear 

kK  K 

extension  of  x  ,  k  =  0,1,  ...,  K  to  [0,T]  .  Finally,  let  x  (t)  be  the 

V  Av  v 

solution  of  (lb,  lc)  with  u(t)  *  u  (t)  .  Then  we  have  j  |x  (t)  -  x  (t)  |  |  -*■  0 
uniformly  for  t  e  [0,T]  as  K  -*•  °°  . 


Remarks : 

Cullum  established  the  uniformity  result  in  the  case  of  autonomous  systems. 
Theorem  1  generalizes  the  result  to  nonautonomous  systems.  The  proof  will  be 
given  in  the  form  of  several  lemmas. 

As  a  preliminary  we  discuss  briefly  the  norms  we  will  use  for  matrices.  Let 
| |x| |  be  any  norm  on  En  and  let  A  be  an  n  *  n  matrix.  Then 
j  | A |  |  =  sup  •  This  norm  has  all  the  norm  properties,  additionally  if 


C  =  AB  then 


jlc  "fa 

|  C | |  <  i  I A ]  1  |  |b| |  .  Some  examples  of  how  this  works  is 


■  (i  >:)' 


j  A | [ 2  =  /x  where  X  is  the  eigenvalue  of 


A  A  with  largest  value. 


(ii)  |  [X|  L  -  [  |X  |  then  |  |aM*  =  Max  [  |a  |  . 

j  i  J 

C±ii)  ||xi|aj  =  Max  |x  |  then  ||  A  |  |  *  =  Max  ^  U.J  • 

i  i  j  3 


In  what  follows  any  pair 


Note  that 


X 

will  be  used  unless  otherwise  specified. 


mi.  ;  llxllj  ;  n||x||j 


I W I. ;  Hxll2 :  ^llxll 
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To  simplify  the  notation  we  will  drop  the  super  *  slnee  It  Is  clear  from 
the  context  which  norm  is  meant  between  ||  ||  ancj  |  |  |  |* 

Let  A(t)  be  continuous  on  [0,T]  and  consider  ^  .  A(t)x(t)  with 
initial  condition  x(0)  -  £°  .  For  K  -  1,2,  ...  let 


k+1  k  /  \ 

s-^s-  -  *  4k 


t1  ♦  i  *(?)]*' 


with  x  «  £°  .  Then  by  successive  substitutions 


k  .  I 
x  *  4>  ,xJ 
kj 


where  for  k  >  j 


^  "  f1  +  l  A(k  K  1 T)]  [J  +  T)]  •••  [I+|^t) 


and  <P  =  I  .  in  particular  we  have 


*“  -  <  5°  • 
k,o 


Lemma  1 : 


.Pc 

Vk,j  is  uniformly  bounded  for  0  <  j  <  k  < 


Proof: 


Since  A  is  continuous  | | A(t) | |  <  A  for 


some  number  A  for  all  t  e  [0,T] 


Then  |  I  [i  +  K  ij]  |  j  <  1  +  I  a  for  all  k  -  0,1 . K  .  Thus 

I  l*kj  I  I  *  I  f1  +  f  A|  |k  j  <  |  |i  +  1  a|  |K  for  all  0  <  j  <  k  <  K  Bt 


1 1  .  I  t |  | K  „  AT 
I  i  +  K  A| |  <  e  .  Let 


itefe'to  is  ■ « -  -*!v  -'t  t !  *V»  -j  «**  .« j.-  , 
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*  (t;s)  -  [i  +  (t  -  kT)A(kT)]  [i  +  I  T)J 

•  •  •  f1  +  f  A(1fA  t)]  f1  +  (-Hr1 1  -  .)iW] 


k  +  1 


K  ^  ■  C  ■  k  ^  an^ 


T  >  s  >  •,^- 


K  «  s  «  k  T  ’  *  is  the  piecewise  linear 


continuous  extension  of  4>  to  [0,T] 


Corollary 


k.  . 

9  U;s'  is  uniformly  bounded  for  0  <  s  <  t  < 


Lemma  2 : 


Consider  B  -  I  +  A  ,  where  A  is  an  arbitrary  fixed  matrix.  Then  there 
exists  io  such  that  ||a||,»#  implies  B  is  nonsingular.  Moreover  far 
every  c  >  0  there  exists  *(«,  >  0  such  that  for  |  |A|  |  <  »(,)  ,  ||,-I  .  „ 


<  e  . 


Proof: 


B  is  nonsingular  if  and  only  if  its  dete™i„a„t  is  nontero.  The  determinant 
Of  a  matrix  is  a  continuous  function  of  its  elements.  For  |  |a|  |  .  0  ||B||.  x 

hence  there  exists  A.  such  that  ||A||  <  Aq  implies  ||B||>  0  .  Similarly  the 
inverse  of  a  matrix  is  a  continuous  function  of  its  elements  wherever  it  is  non¬ 
singular;  in  particular  for  1 1 A  |  |  si  .  FinaHy  I  |  b-1  -  T  I  |  , 

o  y  I  Ml  is  a  continuous 

function  of  the  elements  of  A  for  |  |*|  |  <  ^  .  1 1„-1  -  I J  J  _  1 1  r  _  z  |  j  _  „ 

for  11*11  -0  hence  for  some  *<e>  >  0  |  |*|  |  <  A(e)  implies  |  |  B’1  -  1 1  |  <  e  . 


Lemma  3: 


For  any  t  >  0  there  exists  K  such  that  K  >  K  implies 


A  xK 


I  *  (t;s)  -  A(t)$N(t;s) 

£  •  f . *}. 


uniformly  for  s  e  [0.T]  and  t  e  [0,T]  - 
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Proof : 


£At;s)  -a(|t)[i+  (t  -|i)a(|t)]  V(t;S)  .  !  |{c  -  ^  T)*(kT>  |  |  if  A 

hence  for  any  e  >  0  there  exists  such  that  K  >  implies 

I  |a(|  t| 4K(t;s)  -  a(|  t)[i  +  (t  -  |  t)a(|  t)  <t>K ( t ; s )  [  | 

<  |  |a(|  t|(i  -  [i  +  (t  -  |  t)a(|  t)]  )oK(t;s)  |  | 

<  ||a(|t)||  III  -  [i  +  (t  -|t)a(|t)]  II  ||$K(t;s)|| 


|  |$K(t;s; 


using  Lemma  1  and  2  and  the  fact  that  |  | A ( t ) | |  is  bounded.  Since  A(t)  is 

continuous  on  a  compact  set  [0,T]  it  is  uniformly  continuous  hence  there  exists 

2  2 
K  such  that  K  >  K  implies 


I!a(!t)  -A<t)||  for 

hence  | \a(~  lj$K(t;s)  -  A(t)$K(t;s) | |  <  y  where  $  is  the  upper  bound 


$  (t;s)  given  by  the  corollary  to  Lemma  1.  Thus 


4>K(t;s)  -  A(t)4K(t;s)|  |  = 
-  A(t)$K(t ;s)  |  | 


I  |a(|  t)  [i  +  ( 

|  |A^|  l)oK(t;s) 


I  |a(k  t)  [i  +  (c  -|T)A(lT)]  1?K<tis) 

-|l)A(fl)]'V(t;S)  -A(|T)SK(t;S) 


-  A(t)Ht;s 


<  L  +  £•  =  E 
-22 


Lemma  4: 


Let  X(x,t)  be  continuous  on  E  x  [0,T]  and  satisfy  a  Lipshitz  condition; 


iV»t.  ii*  -t!  t- 
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t 


i.e.,  ||x(x,t)  -  X Cy » t ) 1 1 2  <  L |  | x  —  y 1 1 2  for  all  t  e  [0,T]  ,  and  suppose  x(t) 

solves 

d t  "  X(x*fc) 

X(0)  -  5°  . 

Suppose  y(t)  is  continuous  and  has  a  continuous  derivative  except  at  a  finite 
number  of  simple  discontinuities  and  satisfies 

y (0)  -  x(0)  -  £° 

|  y(t)  -  X(y(t)  * t )  |  1 2  <  e 

for  t  e  [0,T]  except  at  the  discontinuities  of  y(t)  then 
I |y(0  -  x(t) | |2  <  f  {eLt  -  1)  <1  {eLT  -  1}  . 

Proof: 

Birkhoff  and  Rota,  p.  165,  [3]. 

Lemma  5 : 

For  any  e  >  0  there  exists  K  such  that  K  >  R  implies 
|  | d> ( t ; s )  -  $k(t;s)  |  <  e  for  all  0  <  s  <  t  <  T  . 

Proof: 

Let  x(t)  be  the  jth  column  of  1>(t;s)  .  x(t)  solves 


~  -  A(t)x (t) 
x (s)  -  UJ 


where 


u? 


({ii> 


■ 
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K  K 

Let  y  (t)  be  the  jth  column  of  ♦  (t;s)  .  X(x,t)  ■  A(t)x  is  Lipshitz  with 
constant  A  .  By  Lemma  3  for  any  5  >  0  there  exists  such  that  K  >_ 

implies 

6  1  I  $K(t;s)  -  A (t) 4>K( t ; s )  |  |  >_  |  |^-  y  (t)  -  A(t)y(t)  |  | 

uniformly  in  0<s<t<T  except  where  ^7  y(t)  is  discontinuous.  Thus  by 
Lemma  4,  |  |y(t)  -  x ( t ) | [ ^  ^  {eLT  -  1}  .  If  we  take 


6  < 


Le 


,  LT 

v(e  -  1) 


then 

| | y ( t)  -  x ( t ) | l2  <  J 

or 

|||t(t;s)-'{'^(t;s)||  <  e  for  all  0  *  s  <  t  <  T  • 

Lemma  6 : 

Consider  the  system 


—  -  A(t)x(t)  +  B(t)uft) 

x(0)  =  C° 

k  k  +  1 

for  any  u(t)  which  is  constant  on  the  intervals  —  T  <  t  <  — - —  T  ,  k  =  0,1,  . . . ,  K-l 

In  *  l\ 

and  is  uniformly  bounded  in  norm  by  U  (i.e.,  ||u(t)||  <  U  for  all  t  e  [ 0 ,  T  ] ) 

where  B  is  continuous  and  ||B(t)jj  <  B  for  all  t  e  [0,T]  ,  A(t)  is 
continuous  and  | | A ( t )  |  |  <  A  for  all  t  e  [0,T]  .  Let  x(t)  be  a  solution  of 
this  differential  equation. 


Also  consider  the  approximating  system 


-  *(H‘k  ♦  *(H“k 


O  ,0 

X  -  £ 


where  u  -  u  .  Let  x  ,  k  •  0,1,  . ...  K  be  the  solution. 

Then  for  all  e  >  0  there  exists  K  such  that  K  >  K  Implies 

for  k  -  0,1,  . . . ,  K 


I  T)  “  xk|  I 


<  c 


uniformly  for  all  u  satisfying  the  above  conditions. 


Proof: 


k  „ 
KT 


C(k  T)  *  4(|  T‘°)^°  +  {  *(!  T ,sj  B(s) u  (s)ds 


i±l 

K 


||t,0^°+  l  J  *(k  T,8|b(s)u^<1s  . 
J‘°  iT 


K 


On  the  other  hand  by  successive  substitution 


>1 

k,o 


V  /  I  B(i  i  u3\ 

5i0  k,j+l  K  \K  u  ) 


K(f  T,0)5° + 1  *(£  T’J~ri  T)i  b(kt)u1 


Hence 
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Immediately  by  Lemma  5  the  first  term  can  be  made  arbitrarily  small.  We  non-  turn 
to  the  second  term.  Since  4  and  B  are  continuous  they  are  uniformly  continuous 

T,sJb(s)||  can  be  made  uniformly 

small  in  j  for  any  T  <  s  <  ^  T  for  K  sufficiently  large.  By  Lemma  5 

1\  B  *  K 

lh(|  t,s)b(s)  -  *k(|t,s)b(s)||  <  ||*(|t,s)  -  /(|t,s)|!b 
can  also  be  nade  uniformly  small.  Thus 

||*(f  T,s)B(s)  -)b(J;  l)|  |  J 

||«(|t,s)b(s)  -  *K(!  T,s|b(s)||  +  T.s)b(s)  -  »K(!  t)B(|  T)l! 

can  be  made  arbitrarily  small  for  K  sufficiently  large.  Since  ||u^j|  <  U 
the  result  follows. 


[0.T1  thus  I  hK(|  T.J-  K-i  l)B(i  l)  -  /(| 


Proof  of  Theorem  1: 

The  theorem  .iollows  directly  from  Lemma  6  and  the  fact  that  the  x^‘(t) 
and  x^(t)  are  uniformly  continuous  on  [0,T]  . 


The  results  given  here  generalize  readily  to  the  case  where  the  initial 
point  x(0)  ,  and  terminal  point  x(T)  are  allowed  to  range  over  compact  convex 
sets,  Dantzig  and  Van  Slyke,  Section  4,  [8]. 
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7.  EXAMPLES 

In  the  previous  sections  we  generate  a  sequence  of  controls  {u  }  such  that 
we  can  pick  subsequences  which  converge  weakly  or  strongly  to  the  optimal  solution 
of  the  system  under  consideration.  In  general,  if  the  sequence  {u  }  is  an 
arbitrary  minimizing  sequence,  then  it  is  not  always  true  that  it  will  converge 
strongly  to  the  optimal  control  of  the  system.  Conditions  for  strong  convergence 
were  given  in  Theorem  4.2.  Below  we  give  two  examples  in  which  arbitrary  minimizing 
sequences  only  converge  weakly,  Levitin  and  Polyak  [16]. 

Example  1: 


x 

J 


Minimize  1  [x(t)]  dt 

0 

Subject  to  =  u 

I u(t> j  <  i 

x(0)  =  0  . 


for  all  t  z  [0,1] 


*  k 

The  optimal  control  is  u  =  0  .  The  sequence  u  (t)  =  sin  kt  is  a  minimizing 

* 

sequence  but  converges  only  weakly  to  the  optimal  solution  u  =  0  . 


Example  2: 


Minimize  [x(l)]' 


Subject  to 


dx 

dt 


=  u 


J- 


(t)Tdt  <  l 


x  (0)  =■  0 


*  k 

Again,  the  optimal  solution  Ls  u  ^  0  .  The  sequence  u  (t)  =  sin  kt  is  a 

•k 

minimizing  sequence  but  converges  only  weakly  to  the  optimal  solution  u  0 


i 
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It  is  obvious  that  it  canf't  converge  strongly  to  the  optimal  solution  u  =  0  . 

Next  we  give  an  example  of  linear  optimal  control  system  and  apply  the 
algorithm  proposed  in  this  paper.  We  actually  solve  the  problem  on  the  computer 
by  discretization  as  given  in  Section  6. 


Minimize  x  (2) 
o 


dx  (t) 

Subject  to  — — 


dx^t) 


x^t) 


T  -  2.0 


u(t)  e  U(t)  <->  1  u ( t )  |  <_  1  for  all  t  e  [0,2] 


x(t)  e  X(t)  <=> 


0  -11  xQ(t)  f  t/2  +  0.125 


.0  -lj  x^t)  L-t/2  +  0.875. 


for  all  t  c  (0,2) 


>  0)  =  (2,0) 


x(2)  =  (xq(2)  ,0.5) 


For  the  above  problem,  we  have 


$(t,T) 


1  t  -  a" 

0  1 


and  hence 
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Minimize  x  (2) 

o 


dx  (t) 

Subject  to  — — —  -  x^(t) 


dx^t) 


dx2(t) 

dt 


x(0) 

x(2) 


u(t) 


-(^(t)  +  ir2(t))x1(t)  +  71^^ (t) y  -  0.125^ 

+  ^2(t)(f  "  °'875) 


(2,0,0) 


(xo(2),ij,x2(2))  ,  x2(2)  <  0  . 


We  again  solve  this  system  and  in  this  way,  obtain  a  sequence  of  optimal 
solutions  as  given  in  Section  4. 

Solution  by  Discretization 

Now,  we  solve  the  above  problem  by  direct  discretization  as  mentioned  in 
Section  6.  We  solve  the  system  for  two  values  of  number  of  discretizations. 

1)  Let  K-10,A-~-i 


A(kA)  - 


B(kA) 


'0  1 

P  0. 
O' 

UJ 


k  -  0,1,  . . . ,  10 


k  -  0,1 . 10 


/ 

'0  -1' 

k 

x 

5  +  0,125 

)  k 

0 

( 

\x 

< 

> 

.0  -i 

- , 

-  |  +  0.875 
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nk  ;  k 
U  *=  <u 


V 

k 

u  <_ 

Lh 
_ Li 

_1_ 

k  =  0,1 . 9 


Define  <t[j,k]  =  [I  +  AA((k  -  1)A)][I  +  AA((k  -  2)  A)  ]  ...  [I  +  AA  (j  A)  ]  for  j  <  k 
k  -  0,1,  ...»  K.  and  "t*  [  J ,  j  ]  *  I  for  all  j  .  Then 


*[J,k] 


1  1/5 


.0  1  J 


-.k-j 


k  >  j 


and 


k-1 


xk  -  $[0,k]x°  +  l  4[j  +  1 ,k]AB(jA)u^ 


and  the  linear  programming  formulation  is 

£ 

Minimize  x 


’  K 

X 

o 

9 

Subject  to 

-  1 

0 

•  — 

j=o 

(9  -  j )  / 2  5 


1/5 
1  <  uJ  <  1 


l-ij 


j  “  0,1,2,  ....  9 


At  the  kth  iteration,  to  generate  cuts  to  be  added  to  the  above  formulation,  we 
generate  tt  *  as 


k-1 


1  if  l  uj(k_1)  +  0.5k  +  0.125  <  0 

J-0 


0  otherwise 


k-1 


[1  if  l  uj (k_1)  +  0.5k  +  4.875  <  0 
j-0 


0  otherwise 


where  u^k  ^  j  -  0,1,  ...,  9  is  the  optimal  control  at  the  (k  -  l)<.st  iteration, 
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For  the  next  iteration,  the  new  equation  to  be  added  to  the  previous  mathematical 
programming  formulation  is  given  by 


10 


I 

k-1 


k  k 
1T1,7T2 


l  uJ  -  0.5k  -  0.125 

j=0 


l  uJ  +  0.5k  -  4.875 
j-0 


<  0 


or 


10  /  ,  .  v  k-1  10  . 

I  'V'S  I  ^  z  l  irv(0. 5k  +  0.125) 
k-1  '  '  j-0  k-1 

10  w 

+  l  it, (-0.5k  +  4.875)  . 
k-1 


In  the  above  case,  we  obtain  the  optimal  solution  of  the  discrete  problem 
in  four  iterations  as  shown  in  Figure  2  through  5.  The  violation  of  state  space 
constraints  at  various  iterations  is  shown  by  crossed  lines. 


2)  Similarly,  let  K 


20  then  A  *  —  .  In  this  case,  we  have 


*[j,k]  - 


1  1/10 


Lo  i 


k-j 


k  >  j 


and  the  corresponding  linear  program  is 


Minimize 

o 


20 

X 

0 

19 

"(19  -  j )  / 100" 

uJ  - 

’  2" 

Subject  to 

-  1 

0 

•  • 

J-o 

1/10 

.-4. 

-  1  <  uJ  <  1 


j  -  0,1,2, 


19 
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\ 


and  in  order  to  generate  cuts  at  the  kth  iteration,  we  generate  it 
k  -  1,2,  ...»  19  as  follows: 


k 

it. 


1  if 


l  uJ(k"1)  +  0.5k  +  0.75  <  0 

j-o 


otherwise 


k-1 


[l  if  I  u 

[0  otherwise 


j (k-1) 


-  0.5k  +  9.25  0 


and  the  new  equation  to  be  added  to  the  previous  mathematical  programming  formula¬ 
tion  is  given  by 


l  uJ  -  0.5k  -  0.75 
J-0 


l  +  0.5k  -  9.25 
j-l 


0 


or 


20 

l 

k=l 


<  y  TTk C0 . 5k  +  0.75) 
k-1 


20  . 

+  l  TT,(-0.5k  +  9.25)  . 
k-1 


In  this  case,  we  obtain  optimal  solution  in  six  iterations  as  shown  in 


Figures  6  through  11. 


State  Space  Constraints 
-xx(t)  <_  t/2  +  0.125 


OPT  DIAL  TRAJECTORY  WITHOUT  STATE  SPACE 
CONSTRAINTS  FOR  CONTINUOUS  SYSTEM 


N  *  11 


Iteration  No.  1 


FIGURE  2 
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t 


t) 


N  -  21 


Iteration  No.  1 


FIGURE  6 


FIGURE  8 
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